/home/tony/Work/glockenspiel/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend("sox_io")
2023-10-05 18:14:26 | WARNING | root | Pytorch pre-release version 2.1.0.dev20230831+cu118 - assuming intent to test it
/home/tony/Work/glockenspiel/s3prl/s3prl/run_downstream.py:157: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend('sox_io')
[Featurizer] - Take a list of 13 features and weighted sum them.
[Featurizer] - The selected feature hidden_states's downsample rate is 960
[Runner] - Start a new experiment
mert_25hz_demuc_6x {'model_config': None, 'refresh': False}
Loading model from Local mert_25hz_demuc_6x
load local model succeed
load local feat extractor
Wav2Vec2FeatureExtractor {
  "do_normalize": false,
  "feature_extractor_type": "Wav2Vec2FeatureExtractor",
  "feature_size": 1,
  "padding_side": "right",
  "padding_value": 0,
  "return_attention_mask": true,
  "sampling_rate": 24000
}


overall:   0%|          | 0/4000 [00:00<?, ?it/s]

train:   0%|          | 0/37 [00:00<?, ?it/s]
overall:   0%|          | 1/4000 [00:02<2:24:56,  2.17s/it]

train:   3%|▎         | 1/37 [00:02<01:18,  2.17s/it]
overall:   0%|          | 2/4000 [00:02<1:26:52,  1.30s/it]

train:   5%|▌         | 2/37 [00:02<00:45,  1.30s/it]
overall:   0%|          | 3/4000 [00:03<1:07:10,  1.01s/it]

train:   8%|▊         | 3/37 [00:03<00:34,  1.01s/it]
overall:   0%|          | 4/4000 [00:04<57:43,  1.15it/s]  

train:  11%|█         | 4/37 [00:04<00:28,  1.15it/s]Process Process-8:
Process Process-12:
Process Process-7:
Process Process-5:
Process Process-4:
Process Process-6:
Process Process-10:
Process Process-3:
Exception ignored in: <function _MultiProcessingDataLoaderIter.__del__ at 0x7ff5de764160>
Traceback (most recent call last):
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1478, in __del__
    self._shutdown_workers()
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1442, in _shutdown_workers
    w.join(timeout=_utils.MP_STATUS_CHECK_INTERVAL)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/multiprocessing/process.py", line 149, in join
    res = self._popen.wait(timeout)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/multiprocessing/popen_fork.py", line 40, in wait
    if not wait([self.sentinel], timeout):
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/multiprocessing/connection.py", line 931, in wait
    ready = selector.select(timeout)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/selectors.py", line 416, in select
    fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt: 

train:  11%|█         | 4/37 [00:19<02:41,  4.88s/it]
Traceback (most recent call last):
  File "/home/tony/Work/glockenspiel/s3prl/s3prl/run_downstream.py", line 219, in <module>
    main()
  File "/home/tony/Work/glockenspiel/s3prl/s3prl/run_downstream.py", line 215, in main
    eval(f'runner.{args.mode}')()
  File "/home/tony/Work/glockenspiel/s3prl/s3prl/downstream/runner.py", line 292, in train
    features = self.upstream.model(wavs)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/tony/Work/glockenspiel/s3prl/s3prl/upstream/mert/expert.py", line 138, in forward
    output_values = self.model(**input_values, output_hidden_states=True)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/tony/Work/glockenspiel/suno_utils/suno_utils/models/mert/modeling_MERT.py", line 173, in forward
    attention_mask = self._get_feature_vector_attention_mask(
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/transformers/models/hubert/modeling_hubert.py", line 926, in _get_feature_vector_attention_mask
    attention_mask = attention_mask.flip([-1]).cumsum(-1).flip([-1]).bool()
KeyboardInterrupt

overall:   0%|          | 4/4000 [00:19<5:29:01,  4.94s/it]